[ntuple] Make RNTupleChainProcessor composable#17393
Merged
enirolf merged 8 commits intoroot-project:masterfrom Feb 9, 2025
Merged
[ntuple] Make RNTupleChainProcessor composable#17393enirolf merged 8 commits intoroot-project:masterfrom
RNTupleChainProcessor composable#17393enirolf merged 8 commits intoroot-project:masterfrom
Conversation
Test Results 18 files 18 suites 4d 13h 53m 5s ⏱️ For more details on these failures, see this check. Results for commit 3da39b3. ♻️ This comment has been updated with latest results. |
enirolf
force-pushed
the
ntuple-processor-chain-composition
branch
2 times, most recently
from
8d818f5 to
75745f8
Compare
enirolf
force-pushed
the
ntuple-processor-chain-composition
branch
2 times, most recently
from
bf4b538 to
4b69b26
Compare
3 tasks
silverweed
requested changes
Jan 28, 2025
Contributor
silverweed
left a comment
silverweed
left a comment
There was a problem hiding this comment.
I left a couple of, mostly minor, comments.
In general lgtm but there are some changes introduced in one commit that are changed by later commits, so it's a bit hard to review commit-wise.
enirolf
force-pushed
the
ntuple-processor-chain-composition
branch
from
4b69b26 to
b538fcc
Compare
silverweed
approved these changes
Jan 31, 2025
Contributor
|
lgtm but better wait for other people's approval as well |
jblomer
reviewed
Jan 31, 2025
Contributor
jblomer
left a comment
jblomer
left a comment
There was a problem hiding this comment.
That's great! I have some questions on details.
Prevents the page sources corresponding to the processor to be openened upon creation. Instead, defer opening them until the first `Advance` call.
hahnjo
reviewed
Feb 3, 2025
enirolf
force-pushed
the
ntuple-processor-chain-composition
branch
from
b538fcc to
e70afa1
Compare
hahnjo
reviewed
Feb 6, 2025
In anticipation of the composition of the different processors (in particular, the join processor), entries loading should offer the possibility for random access. This will not change anything about the way users can interact with the processor, which is exclusively through the (linear) iterator.
With this change, the `RNTupleChainProcessor` iterates over other `RNTupleProcessor` objects instead of individual ntuples. This allows us to chain, for example, ntuples that have previously been joined using the `RNTupleJoinProcessor`.
When no model is provided, it needs to be inferred from the page source, which first has to be created. This model inference previously happened at the `RNTupleProcessor::Create` factory method. Deferring this to the `RNTupleSingleProcessor` constructor instead removes the need to create the page source twice. In addition, when a model is provided, the page source only gets created when the processor is actually connected.
It is currently not used, and can be re-added when we actually start using it.
enirolf
force-pushed
the
ntuple-processor-chain-composition
branch
from
e70afa1 to
3da39b3
Compare
hahnjo
approved these changes
Feb 7, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR introduces the possibility to create
RNTupleChainProcessors from other processor objects. In turn, this makes it possible to, for example, create a chain of joined ntuples.This PR is part of a bigger set of (foreseen) changes, collected and tracked in #17132.